IMAGE PROCESSOR AND MONITORING SYSTEM 

BACKGROUND OF THE INVENTION 

The present invention relates to an image processing 
5 technique for generating a synthesized image from images 
taken with a plurality of cameras. In particular, the pre- 
sent invention relates to a technique that can be effectively 
applied to monitoring systems used as an assist in ensuring 
safety driving of a vehicle and the like. 

10 In recent years, with widespread use of car-mounted dis- 

plays and cost reduction in video equipment such as cameras, 
devices for monitoring the surroundings of a vehicle with 
cameras to assist safety driving have been commercialized and 
are now becoming popular. 

15 As an example of such conventional vehicle surroundings 

monitoring devices, the following construction is known. 
That is, a camera is placed at a position on a vehicle where 
an image of a blind area for the driver, such as the rear of 
the vehicle, can be taken with the camera, and the image 

20 taken (camera image) is shown on a display for monitoring. 

For example, Japanese Laid-Open Patent Publication 
No. 58-110334 discloses the following construction. A plu- 
rality of cameras are placed at different positions on a ve- 
hicle. A plurality of camera images taken with the cameras 

25 are modified and synthesized, to generate a synthesized image 
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as is viewed from the above of the vehicle with the vehicle 
in the center and display the synthesized image on an in-car 
display* Using this vehicle surroundings monitoring device, 
the driver sitting on the driver's seat can monitor on the 
5 display a blind area of which direct view is not possible. 
This contributes to avoiding an accident that would otherwise 
have occurred/ and also facilitating the driving. 
[Problems to be solved] 

However / the inventors of the present invention have 

10 found from experiments and examinations that the conventional 
construction has the following problems • 

FIGS. 22A to 22C illustrate examples of image synthesis 
performed when a vehicle is moving. In the case of using 
cameras for interlaced scanning, an input image as shown in 

15 FIG. 22A is obtained when the vehicle is moving. That is, 
every other horizontal line is displaced resulting in comb- 
shaped appearance. This input image is not particularly 
Strange as long as it is displayed as it is on a display for 
interlaced scanning. However, when this input image is modi- 

20 fied for synthesis, the comb-shaped displacement is also 
modified during the image modification as shown in FIG. 22B, 
and this causes a lag between the image-taking timing and the 
displaying timing. The resultant image makes the observer 
feel strange. That is, so-called interlace noise appears on 

25 the synthesized image more significantly. 
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The unnaturalness as described above does not appear 
when a one-field synthesized image is generated from one- 
field input images. However, the resolution of an input im- 
age is low in this per-field synthesis, compared with per- 
5 frame synthesis. Therefore, when an image of two fields is 
observed as a frame image as shown in FIG. 22C, comb-shaped 
displacement occurs due to an error in quantization of coor- 
dinate values even when the image is still. This deterio- 
rates the image quality, especially in the area of a 

10 synthesized image where an input image has been enlarged, 
when the vehicle is standing still. 

That is, as the first problem, when a one-frame synthe- 
sized image is generated from one- frame (two-field) input im- 
ages, the synthesized image is unnatural when the vehicle is 

15 moving and when a moving image is taken. When a one-field 
synthesized image is generated from one-field input images, 
the resolution of the synthesized image is low. 

As the second problem, in modification of a camera im- 
age, the input image may be contracted depending on the com- 

20 position of a synthesized image. As a result, aliasing 
distortion may partially be generated in the synthesized im- 
age. 

As the third problem, the junctures of input images on a 
synthesized image may be unnatural. For example, because 
25 white balancing and the like are adjusted separately for the 
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respective cameras, the brightness and the tint may fail to 
match at the junctures as shown in FIG. 23A. Also, in some 
image synthesis methods, the junctures of images may be dis- 
placed due to calculation error, erroneous detection of the 
5 position of an object, and the like, as shown in FIG. 23B. 

SUMMARY OF THE INVENTION 

An object of the present invention is providing an image 
processor for generating a synthesized image from a plurality 

10 of camera images, which can generate a more natural synthe- 
sized image compared with the conventional processors. 

Specifically, the image processor of the present inven- 
tion, which receives camera images taken with a plurality of 
cameras capturing the surroundings of a vehicle and generates 

15 a synthesized image from the camera images, includes a pa- 
rameter generation section constructed to be able to generate 
a plurality of image synthesis parameter groups each repre- 
senting the correspondence between the camera images and the 
synthesized image and having different spatial or temporal 

20 resolution relations, wherein the synthesized image is gener- 
ated from the camera images according to the image synthesis 
parameter group generated by the parameter generation sec- 
tion, and the parameter generation section switches the image 
synthesis parameter group to be generated according to an 

25 output of a vehicle motion detection section for detecting 
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the motion of the vehicle. 

According to the invention described above, it is possi- 
ble to switch the image synthesis parameter group used for 
generation of a synthesized image according to the motion of 
5 the vehicle. This makes it possible to reduce the unnatural- 
ness of the synthesized image observed when the vehicle is 
moving and when an image of a moving object is taken, and 
also prevent the reduction in resolution observed when the 
vehicle is standing still. 

10 In the image processor of the present invention, the pa- 

rameter generation section preferably includes: a paramieter 
storage section for storing the plurality of image synthesis 
parameter groups; and a parameter selection section for 
selecting at least one among the plurality of image synthesis 

15 parameter groups stored in the parameter storage section ac- 
cording to the output of the vehicle motion detection sec- 
tion. 

In the image processor of the present invention, pref- 
erably, the camera images are interlaced images, and the plu- 

20 rality of image synthesis parameter groups include at least a 
frame-base image synthesis parameter group and a field-base 
image synthesis parameter group. Preferably, the parameter 
generation section generates the field-base image synthesis 
parameter group when the motion of the vehicle detected by 

25 the vehicle motion detection section is relatively fast, and 
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generates the frame-base image synthesis parameter group when 
the motion of the vehicle is relatively slow. 

In the image processor of the present invention^ the ve- 
hicle motion detection section preferably detects the motion 
5 of the vehicle from the camera images . 

In the image processor of the present invention, pref- 
erably, the plurality of cameras are constructed to be able 
to. switch a capture pattern according to an input switch sig- 
nal, and the parameter generation section sends the switch 

10 signal to the cameras, together with generating the image 
synthesis parameter group, according to the output of the ve- 
hicle motion detection section, to switch the capture pat- 
terns of the cameras. Preferably, the parameter generation 
section selects the image synthesis parameter group and 

15 switches the capture patterns of the cameras according to an 
output of a vehicle status detection section for detecting 
brightness of the surroundings of the vehicle, in addition to 
the output of the vehicle motion detection section. 

In the image processor of the present invention, pref- 

20 erably, the parameter generation section generates the image 
synthesis parameter group according to an output of a vehicle 
status detection section for detecting the vehicle status 
such as an operation by a driver of the vehicle or whether or 
not an obstacle exists in the surroundings of the vehicle, in 

25 addition to the output of the vehicle motion detection sec- 
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tion. 

The monitoring system of the present invention includes 
the image processor described above as an image processing 
section. 

5 Alternatively, the image processor of the present inven- 

tion , which receives camera images taken with a plurality of 
cameras capturing the surroundings of a vehicle and generates 
a synthesized image from the camera images, includes: a pa- 
rameter storage section for storing a plurality of sets of an 

10 image synthesis parameter group representing the correspon- 
dence between the camera images and the synthesized image and 
a filter parameter group corresponding to the image synthesis 
parameter group; a parameter selection section for selecting 
at least one among the plurality of sets of the image synthe- 

15 sis parameter group and the filter parameter group stored by 
the parameter storage section according to an output of a ve- 
hicle motion detection section for detecting the motion of 
the vehicle and an output of a vehicle status detection sec- 
tion for detecting the status of the vehicle such as an op- 

20 eration by a driver of the vehicle or whether or not an 
obstacle exists in the surroundings of the vehicle; and a 
filtering section for performing frequency band IdLmitation 
filtering for the camera images according to the filter pa- 
rameter group of the set selected by the parameter selection 

25 section, wherein the synthesized image is generated from the 
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camera images filtered by the filtering section according to 
the image synthesis parameter group of the set selected by 
the parameter selection section. 

According to the invention described above, the filter 
5 parameter group is selected according to the motion and 
status of the vehicle, and the camera images are subjected to 
frequency band limitation filtering according to the selected, 
filter parameter group. This effectively suppresses the 
aliasing distortion in the synthesized image. 
10 In the image processor of the present invention de-^ 

scribed above, the filter parameter group preferably includes 
filtering setting data for each pixel position of the camera 
image. 

The monitoring system of the present invention includes 
15 the image processor described above as an image processing 
section. 

Alternatively, the image processor of the present inven- 
tion, which receives camera images taken with a plurality of 
cameras capturing the surroundings of a vehicle and generates 

20 a synthesized image from the camera images, includes: a 
brightness correction parameter calculation section for cal- 
culating brightness correction parameters for correcting the 
brightness and tint of the camera images; and a brightness 
correction section for correcting the brightness and tint of 

25 the camera images using the brightness correction parameters 
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calculated by the brightness correction parameter calculation 
section, wherein the synthesized image is generated from the 
plurality of camera images subjected to brightness correction 
by the brightness correction section according to an image 
5 synthesis parameter group representing the correspondence be- 
tween the camera images and the synthesized image, the image 
synthesis parameter group includes overlap area data for an 
overlap area on the synthesized image in which coverages of 
the plurality of cameras overlap, the overlap area data indi- 

10 eating coordinates of pixels in camera images corresponding 
to the overlap area, and the brightness correction parameter 
calculation section receives the overlap area data and calcu- 
lates the brightness correction parameters using brightness 
and tint data for the pixels in the camera images correspond- 

15 ing to the overlap area indicated by the overlap area data. 

According to the invention described above, brightness 
correction parameters are calculated using information on the 
brightness and tint of the camera images corresponding to the 
overlap area on the synthesized image, and brightness correc- 

20 tion is performed using the brightness correction parameters* 
This reduces the unnaturalness of the juncture on the synthe- 
. sized image. 

In the image processor of the present invention de- 
scribed above, preferably, the brightness correction parame- 
25 ter calculation section performs statistical processing on 
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the brightness in the overlap area for the camera images cor- 
responding to the overlap area, and calculates the brightness 
correction parameters based on the processing results. 

In the image processor of the present invention de- 
5 scribed above, when a plurality of overlap areas exist, the 
brightness correction parameter calculation section prefera- 
bly sets priorities to the overlap areas to be considered 
during the calculation of the brightness correction parame- 
ters according to an output of a vehicle motion detection 
10 section for detecting the motion of the vehicle. 

In the image processor of the present invention de- 
scribed above, the brightness correction section is prefera- 
bly incorporated in the cameras . 

The monitoring system of the present invention includes 
15 the image processor described above as an image processing 
section. 

Alternatively, the image processor of the present inven- 
tion receives camera images taken with a plurality of cameras 
capturing the surroundings of a vehicle and generates a syn- 

20 thesized image from the camera images, wherein, in an overlap 
area in which coverages of a plurality of cameras overlap on 
the synthesized image, a camera image used for generation of 
the synthesized image is selected among camera images from 
the plurality of cameras according to an output of a vehicle 

25 motion detection section for detecting the motion of the ve- 
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hide or an output of a vehicle status detection section for 
detecting the status of the vehicle such as an operation by a 
driver of the vehicle or whether or not an obstacle exists in 
the surroundings of the vehicle. 
5 Alternatively, the image processor of the present inven- 

tion receives camera images taken with a plurality of cameras 
capturing the surroundings of a vehicle and generates a syn- 
thesized image from the camera images, wherein, in an overlap 
area in which coverages of a plurality of cameras overlap on 

10 the synthesized image, weights to camera images from the plu- 
rality of cameras are set according to an output of a vehicle 
motion detection section for detecting the motion of the ve- 
hicle or an output of a vehicle status detection section for 
detecting the status of the vehicle such as an operation by a 

15 driver of the vehicle or whether or not an obstacle exists in 
the surroundings of the vehicle. 

In the image processor of the present invention de- 
scribed above, the processor includes: an image synthesis pa- 
rameter group associating pixels in the synthesized image 

20 with pixels in the camera images; and a table representing 
the correspondence between a weight reference number and a 
combination of sets of weighting information, wherein a por- 
tion of the image synthesis parameter group corresponding to 
the overlap area holds any of the weight reference numbers 

25 shown in the table. 
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The monitoring system of the present invention includes 
the image processor described above as an image processing 
section. 

Alternatively^ the image processor of the present inven- 
5 tion receives camera images taken with a plurality of cameras 
capturing the surroundings of a vehicle and generating a syn- 
thesized image from the camera images^ wherein the image 
processor includes an image synthesis parameter group associ- 
ating pixels in the synthesized image with pixels in the cam- 
10 era images, and in an overlap area in which coverages of the 
plurality of cameras overlap on the synthesized image, the 
image synthesis parameter group includes weights set to cam- 
era images from the plurality of cameras represented by a 
dithering method. 

15 

BRIEF DESCRIPTION OF THE DRAWINGS 
FIG. 1 is a structural view of a vehicle surroundings 
monitoring system of EMBODIMENT 1 of the present invention. 

FIGS. 2A to 2C are views for description of image syn- 
20 thesis using an image synthesis parameter group. 

FIG. 3 shows an example of selection of image syntheisis 
parameter groups in EMBODIMENT 1 of the present invention. 

FIG. 4 shows an example of selection of image synthesis 
parameter groups and capture patterns in EMBODIMENT 1 of the 
25 present invention. 
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FIG, 5 shows an example of selection of image synthesis 
parameter groups and capture patterns in the case of using an 
output of a brightness detection means in EMBODIMENT 1 of the 
present invention . 
5 FIG. 6 shows an example of selection of image synthesis 

parameter groups in the case of using outputs of a switch de- 
tection means and an obstacle detection means in EMBODIMENT 1 
of the present invention. 

FIGS. 7A and 7B show exemplary compositions of a synthe- 
10 sized image. 

FIG. 8 is a structural view of a vehicle surroundings 
monitoring system of EMBODIMENT 2 of the present invention. 

FIG. 9 is a view showing a filter parameter group. 

FIG. 10 shows an example of selection of image synthesis 
15 parameter groups and filter parameter groups in EMBODIMENT 2 
of the present invention. 

FIGS. IIA and llB are views showing filter parameter 
groups . 

FIG. 12 is a structural view of a vehicle surroundings 
20 monitoring system of EMBODIMENT 3 of the present invention. 

FIGS. 13A to 13D are views showing an image synthesis 
parameter group for generating a synthesized image from a 
plurality of camera images. 

FIGS. 14A and 14B are views showing a synthesized image 
25 including an overlap area and an image synthesis parameter 
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group for generating the synthesized Image. 

FIG. 15 is a view showing the positions at which cameras 
and obstacle sensors are placed in EMBODIMENT 4 of the pre- 
sent invention. 

5 FIGS. 16A to 16C are views showing a synthesized image 

and image synthesis parameter groups for generating the syn- 
thesized image. 

FIGS. 17A to 17C are views showing examples of selection 
of an image synthesis parameter group in EMBODIMENT 4 of the 
10 present invention. 

FIG. 18 is a view showing another image synthesis pa- 
rameter group in EMBODIMENT 4 of the present invention. 

FIG. 19 is a view showing yet another exemplary image 
synthesis parameter group in EMBODIMENT 4 of the present 
15 invention. 

FIGS. 20A and 20B are views showing other image synthe- 
sis parameter groups in EMBODIMENT 4 of the present inven- 
tion. 

FIG. 21 is a view showing another construction for im- 
20 plementing the present invention. 

FIGS. 22A to 22C are views for description of conven- 
tional problems. 

FIGS. 23A and 23B are views for description of conven- 
tional problems. 

25 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Hereinafter, preferred embodiments of the present inven- 
tion will be described with reference to the accompanying 
drawings . 
5 EMBODIMENT 1 

FIG. 1 illustrates a construction of a vehicle surround- 
ings monitoring system of EMBODIMENT 1 of the present inven- 
tion. Referring to FIG. 1, cameras 101 take images ^ A/D 
converters 102 digitalize analog image signals, and frame 
10 memories 103 temporarily hold the digitalized input images. 
Each of the frame memories 103 have a so-called double-buffer 
structure, which temporarily holds immediately preceding one- 
frame image data while continuing to store the images output 
from the camera 1. In other words, the frame memories 103 
15 are constructed to permit read of arbitrary pixel data in the 
immediately preceding one-frame image in response to a read 
request from an image synthesis section 104. 

A vehicle motion detection section 107 detects motion of 
a vehicle. A parameter storage section 109 stores in advance 
20 a plurality of image synthesis parameter groups representing 
the correspondence between camera images and a synthesized 
image. A parameter selection section 108 selects one of the 
plurality of image synthesis parameter groups stored in the 
parameter storage section 109 according to the motion of the 
25 vehicle detected by the vehicle motion detection section 107. 
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The image synthesis section 104 sequentially reads images 
from the frame memories 103 according to the image synthesis 
parameter group selected by the parameter selection sec- 
tion 108 to combine the images and output a synthesized image. 
5 A D/A converter 105 converts the synthesized image to analog 
signals, and a display 106 displays the analog-converted syn- 
thesized image. 

The A/D converters 102, the frame memories 103, the im- 
age synthesis section 104, the D/A converter 105, the parame- 
10 ter selection section 108, and the parameter storage 
section 109 constitute an image processor or an image proc- 
essing section. The parameter selection section 108 and the 
parameter storage section 109 constitute a parameter genera- 
tion section. 

15 In this embodiment, assume that dLmages handled by the 

cameras 101 and the display 106 are interlaced scanning im- 
ages. Also assume that the vehicle motion detection sec- 
tion 107 detects the rotational speed of the wheels of the 
vehicle, the driving direction, and the like as the motion of 

20 the vehicle, from signals sent from sensors placed on an axle 
of a wheel, the shift lever, and the like. 

The system also includes a vehicle status detection sec- 
tion 200, which will be described later together with the us- 
age thereof. 

25 Hereinafter, the operation of the vehicle surroundings 
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monitoring system with the above construction will be de- 
scribed . 

FIGS. 2A to 2C illustrate how to combine images using an 
image synthesis parameter group. FIG. 2A illustrates an ex- 
5 ample of a camera image that is a one-frame image from a cam- 
era 1 placed as shown in FIG. 15. FIG. 2B illustrates an 
example of a one-frame synthesized image obtained by modify- 
ing and combining four camera images. 

FIG. 2C is an example of an image synthesis parameter 

10 group, in which respective sets of coordinates have one-to- 
one correspondence with the coordinates of pixels in the syn- 
thesized image. Each set of coordinates includes parameters 
representing the camera number with which a camera image is 
taken and a two-dimensional array indicating the coordinates 

15 of a pixel in the camera image used to generate the pixel in 
the synthesized image corresponding to the set of coordinates. 
In the example shown in FIG. 2C, information of "camera No. 1, 
coordinates (xi, yi)" is stored in the coordinates (xo, yo) 
in the image synthesis parameter group. This indicates that 

20 the pixel at the coordinates (xo, yo) in the synthesized im- 
age is generated by the pixel at the coordinates (xi, yi) in 
the camera image from the camera 1. In this way, it is pos- 
sible to describe the correspondence between the pixels in a 
plurality of camera images and those in a synthesized image 

25 by use of an image synthesis parameter group constructed as 



17 



shown in FIG. 2C. 

In this embodiment, it is assumed that the parameter 
storage section 109 stores a plurality of image synthesis pa- 
rameter groups having different spatial or temporal resolu- 
tions for at least either the camera images or the 
synthesized image. 

As such a plurality of image synthesis parameter groups 
having different spatial or temporal resolutions^ assume spe- 
cifically that the parameter storage section 109 stores a 
frame-base image synthesis parameter group and a field-base 
image synthesis parameter group having the same composition 
of the synthesized image. In the frame-base image synthesis 
parameter group, a one- frame camera image corresponds to a 
one- frame synthesized image. In the field-base image synthe- 
sis parameter group, the first field of a camera image corre- 
sponds to one field of a synthesized image, while the second 
field of the camera image corresponds to the other field of 
the synthesized image. The field-to-field correspondence be- 
tween the camera image and the synthesized image can be eas- 
ily described by imposing a restriction that the part of a 
camera image having an even y coordinate value corresponds to 
the part of a synthesized image having an even y coordinate 
value and that the part of the camera image having an odd y 
coordinate value corresponds to the part of the synthesized 
image having an odd y coordinate value. 
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FIG. 3 shows an example of the operation of the parame- 
ter selection section 108. Referring to FIG. 3, the parame- 
ter selection section 108 selects the image synthesis 
parameter group according to the vehicle speed detected by 
5 the vehicle motion detection section 107 based on predeter- 
mined criteria. Specif ically^ the parameter selection sec- 
tion 108 selects the frame-base image synthesis parameter 
group when the motion of the vehicle is relatively slow, for 
example, when the vehicle speed is less than 10 km/h, and se- 

10 lects the field-base image synthesis parameter group when the 
motion of the vehicle is relatively fast, for example, when 
the vehicle speed is 10 km/h or more. The image synthesis 
section 104 reads pixel data of camera images from the frame 
memories 103 according to the camera numbers and the coordi- 

15 nate values in the image synthesis parameter group selected 
by the parameter selection section 108, to generate a synthe- 
sized image. The generated synthesized image is converted to 
analog signals by the D/A converter 105 and displayed on the 
display 106. 

20 By the operation as described above, images are combined 

every field when the vehicle speed is high and the motion in 
the camera image is large. This enables generation of a 
natural synthesized image free from comb-shaped displacement. 
On the contrary, images are composed every frame when the ve- 

25 hide speed is low. This enables generation of a clear syn- 
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thesized image with high resolution. That is, by changing 
the image synthesis method according to the motion of the ve- 
hicle, it is possible to reduce the unnaturalness of the syn- 
thesized image that is otherwise observed when the vehicle is 
5 moving, and also prevent the reduction in the resolution of 
the synthesized image that is otherwise observed when .the ve- 
hicle is substantially standing still. 

In this embodiment, the vehicle motion detection sec- 
tion 107 detects the motion of the vehicle, represented by 

10 the rotational speed of the wheels, the driving direction, 
and the like, from signals sent from sensors placed on an 
axle, the shift lever, and the like. Alternatively, the ve- 
hicle motion may be detected from camera images • 

In the case of detecting the vehicle motion from camera 

15 images, images from one of the cameras 101 are sequentially 
read from the frame memory 103, to obtain an image composed 
of a plurality of temporally continuous frames or fields and 
calculate a motion vector in the image. Based on the calcu- 
lated motion vector and camera parameters such as the pre- 

20 measured position, direction, and focal distance of the cam- 
era 101, the direction and speed of the motion of the road 
surface in the image are determined and output as the vehicle 
speed . 

By providing the vehicle motion detection section 107 
25 that detects the vehicle motion from camera imalges as de- 
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scribed above ^ it is no more required to supply inputs other 
than the images, such as the signals from sensors placed on 
an axle and the shift lever. Therefore, a simple system con- 
struction can be realized. 
5 The cameras 101 may have a function of switching a cap- 

ture pattern according to an input switch signal. In such a 
case, the parameter selection section 108 may be constructed 
to switch the capture pattern of the cameras 101, in addition 
to selecting the image synthesis parameter group as described 

10 above, according to the vehicle motion detected by the 
vehicle motion detection section 107. 

FIG. 4 is a view showing an example of the operation of 
the parameter selection section 108 in the case described 
above. In this example, it is assumed that the cameras 101 

15 permit switching between interlaced imaging (one field every 
1/60 second) and non-interlaced imaging (one frame every 1/30 
second) and switching of the shutter speed (exposure time: 
1/30 second, 1/60 second, and 1/120 second). 

As shown in FIG. 4, the parameter selection section 108 

20 selects the image synthesis parameter group according to the 
vehicle speed detected by the vehicle motion detection sec- 
tion 107 based on predetermined criteria, and also switches 
the capture pattern of each camera 101. Specifically, the 
parameter selection section 108 selects the frame-base image 

25 synthesis parameter group when the vehicle speed is less than 
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10 kia/h, or the field-base image synthesis parameter group 
when it is 10 km/h or more. In addition, the parameter se- 
lection section 108 switches the capture pattern of each cam- 
era 101 so that non-interlaced imaging and 1/30-second 
5 exposure time are selected when the vehicle speed is less 
than 10 km/h/ interlaced imaging and 1/60-second exposure 
time are selected when the vehicle speed is between 10 km/h 
or more and less than 30 km/h, and interlaced imaging and 
1/120-second exposure time are selected when the vehicle 

10 speed is 3 0 km/h or more. 

By the operation described above, when the vehicle speed 
is less than 10 km/h, images are taken by non-interlaced im- 
aging and combined per frame. This increases the resolution 
of the synthesized image. When the vehicle speed is 10 km/h 

15 or more, images are taken by interlaced imaging and combined 
per field. This decreases the resolution, but natural motion 
is obtained. In addition, by switching the exposure time 
with the vehicle speed, a natural synthesized image with re- 
duction in blurring due to the motion is obtained. In other 

20 words, a synthesized image with optimum quality determined by 
the motion of the vehicle can be generated by the combination 
of the selection of the image synthesis parameter group and 
the switching of the capture pattern. 

The construction shown in FIG. 1 includes the vehicle 

25 status detection section 200. The outputs of the vehicle 
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status detection section 200 can be used in the selection of 
the image synthesis parameter group and the switching of the 
capture pattern, to further improve the quality of the syn- 
thesized image. The vehicle status detection section 200, 
5 which is constructed of an optical sensor placed on the vehi- 
cle and the like, includes a means 210 for detecting the 
brightness in the surroundings of the vehicle, a means 220 
for detecting the switching operation by the driver, and a 
means 230 for detecting an obstacle in the surroundings of 

10 the vehicle. 

FIG. 5 is a view showing an example of the operation of 
the parameter selection section 108 using the output of the 
brightness detection means 210. In general, the exposure 
time required to take one-frame image by interlaced imaging 

15 is half that required by non-interlaced imaging. In the case 
of a moving image, a longer exposure time causes blurring but 
provides a bright image. On the contrary, a shorter exposure 
time reduces blurring but darkens the image. Based on this 
fact, the parameter selection section 108 performs the selec- 

20 tion of the image synthesis parameter group and the switching 
of the capture pattern of each camera 101 in consideration of 
the brightness in thie surroundings of the vehicle detected by 
the brightness detection means 210, in addition to the vehi- 
cle speed detected by the vehicle detection section 107 . 

25 This enables generation of a synthesized image with optdLmum 
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quality determined by the brightness in the surroundings of 
the vehicle and the vehicle speed. 

FIG,. 6 is a view showing an example of the operation of 
the parameter selection section 108 using the outputs of the 
5 switch detection . means 220 and the obstacle detection 
means 230. Assume that the parameter storage section 109 
stores frame-base and field-base image synthesis parameter 
groups for two types of compositions shown in FIGS. 7A and IB, 
and that the driver is allowed to select one of three options, 

10 "composition A", "composition B", and "automatic", by switch 
operation. The switch detection means 220 detects the switch 
operation by the driver. The obstacle detection means 230 
measures the distance from an obstacle using an ultrasonic 
obstacle sensor placed on the vehicle and outputs the results. 

15 The parameter selection section 108 selects a suitable 

image synthesis parameter group according to the detected 
status of the vehicle, that is, the selection by the driver 
and the distance from an obstacle, and the vehicle speed de- 
tected by the vehicle motion detection section 107. 

20 Composition A of FIG. 7 A includes a wider range of the 

surroundings of the vehicle, while composition B of FIG. 7B 
shows a narrower range of the surroundings of the vehicle in 
an enlarged view. Therefore, supposing the vehicle speed is 
the same, greater comb-shaped displacement is generated in 

25 composition B than in composition A. In view of this fact. 
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the vehicle speed with which the image synthesis parameter 
group is switched from the frame base to the field base may 
be changed according to the motion and status of the vehicle, 
as shown in FIG, 6, This enables generation of a synthesized 
5 image with further optimum quality. 

The vehicle status detection section 200 may also detect 
the operations of the shift lever, the winker, and the like 
by the driver. 

In this embodiment, the field-base and frame-base image 

10 synthesis parameter groups were used as an example. It is 
also possible to use other types of image synthesis parameter 
groups that are different in spatial or temporal resolution 
relation from each other. For example, when a one-frame cam- 
era image is constructed of a plurality of images taken for 

15 different durations, it is possible to use a plurality of im- 
age synthesis parameter groups having different spatial or 
temporal resolution combinations, to provide substantially 
the same effects as those described above. 

It should be noted that the criteria for selection of a 

20 parameter group shown in FIGS. 3 to 6 and 10 are mere exam- 
ples and that a parameter group may be selected according to 
the motion and status of the vehicle based on criteria other 
than the exemplified ones. 

In this embodiment, field-base and frame-base image syn- 

25 thesis parameter groups were prepared. Alternatively, only a 
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frame-base image synthesis group may be prepared to realize 
the operation described above. That is, a field-base image 
synthesis parameter group can be prepared by rounding the y 
coordinate value yi in a camera image to an odd or even num- 
5 ber according to the field to which the y coordinate value yo 
in the frame-base image synthesis parameter group belongs. 
Therefore, only a frame-base image synthesis parameter group 
may be stored in the parameter storage section 109. When a 
frame-base image synthesis parameter group is required, the 

10 parameter selection section 108 reads the frame-base image 
synthesis parameter group as it is. When a field-base image 
synthesis parameter group is required, the parameter selec- 
tion section 108 may convert the frame-base image synthesis 
parameter group stored in the parameter storage section 109 

15 to a field-base image synthesis parameter group in the manner 
described above. 
EMBODIMENT 2 

FIG. 8 illustrates a construction of a vehicle surround- 
ings monitoring system of EMBODIMENT 2 of the present inven- 

20 tion. This construction is different from the construction 
shown in FIG. 1 in that a filtering section 310 is provided 
between the A/D converters 102 and the frame memories 103 to 
perform filtering of camera images. A parameter storage sec- 
tion 330 stores a plurality of sets of an image synthesis pa- 

25 rameter group representing the correspondence between camera 
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images and a synthesized image and a filter parameter group 
corresponding to the image synthesis parameter group. A pa- 
rameter selection section 320 selects a set of the image syn- 
thesis parameter group and the filter parameter group from 
5 the parameter storage section 330 according to the vehicle 
motion detected by the vehicle motion detection section 107 
and the vehicle status detected by the vehicle status detec- 
tion section 200. The filtering section 310 performs fre- 
quency-band limitation filtering for respective camera images 

10 according to the filter parameter group in the set selected 
by the parameter selection section 320. 

FIG. 9 shows an example of the filter parameter group ^ 
which specifies the cut-off frequency for each camera. The 
filtering section 310 performs low-pass filtering with the 

15 cut-off frequency specified in the filter parameter group for 
digitized image data output from the A/D converters 102. 

In the illustrated example^ the cut-off frequency is 
provided for a pixel clock fs, and "l/2fs'' is specified when 
the input image data is to be output as it is and "l/4fs" is 

20 specified when the input image data is to be filtered with a 
cut-off frequency of l/4fs. The cut-off frequency is calcu- 
lated in advance based on a sampling theorem according to the 
contraction rate of the portion of the camera image that is 
most contracted during image synthesis according to the image 

25 synthesis parameter group. For example, when the contraction 
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rate of the portion of a camera image from the camera 1 that 
is most contracted during image synthesis is 1/2 in terms of 
the length, the cut-off frequency for the camera 1 is set at 
l/4fs. 

5 The contraction rate of a camera image can be determined 

using the coordinate values for the camera image in an image 
synthesis parameter group. For example, assume that the pa- 
rameter element at coordinates (xO, yO) in an image synthesis 
parameter group includes "camera No. 1, coordinates (xl, yl)"/ 

10 and the parameter element at coordinates (xO+1, yO) in the 
image synthesis parameter group includes "camera No. I, coor- 
dinates (x2, y2)". In this case, when the distance between 
the coordinates (xl, yl ) and the coordinates (x2, y2) is D, 
the portions at and around the coordinates (xl, yl) and the 

15 coordinates (x2, y2) in the camera image from the camera 1 
have been contracted to 1/D in the synthesized image. In 
this way, it is possible to obtain the contraction rates at 
and around certain pixels in a camera image. The minimum of 
the thus-obtained contraction rates is the contraction rate 

20 of the most contracted portion of the camera image. 

The parameter selection section 320 selects the image 
synthesis parameter group and the filter parameter group ac- 
cording to the vehicle speed detected by the vehicle motion 
detection section 107 and the vehicle status detected by the 

25 vehicle status detection section 200 as shown in FIG. 10, for 
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example. The other operation is- substantially the same as 
that described in EMBODIMENT 1. 

As described in relation to the problems to be solved, 
when an image is contracted in modification, aliasing distor- 
5 tion may be generated in a synthesized image unless a high- 
range frequency component is removed according to the con- 
traction rate. In this embodiment, such aliasing distortion 
can be eliminated by performing the low-pass filtering ac- 
cording to the filter parameter group. Moreover, in the case 

10 of switching the composition of a synthesized image as shown 
in FIG. 7, the filter parameter group may be switched to- 
gether with the switching of the composition. This enables 
generation of a synthesized image with optimum quality with 
reduced aliasing distortion determined by the status and 

15 speed of the vehicle. 

FIGS. IIA and llB are views for description of another 
example of the filter parameter group. The filter parameter 
group shown in FIG. IIB includes parameter elements having 
one-to-one correspondence with the coordinates of pixels in a 

20 one-frame camera image. Each parameter element specifies the 
cut-off frequency for filtering of each camera image. In 
other words, the filter parameter group includes filtering 
setting data for respective pixel positions of each camera 
image. 

25 The parameter selection section 320 selects the image 
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synthesis parameter group and the filter parameter group ac- 
cording to the vehicle speed detected by the vehicle detec- 
tion section 107 and the vehicle status detected by the 
vehicle status detection section 200, and outputs the se- 
5 lected results while performing sequential scanning. The 
filtering section 310 performs low-pass filtering for the 
digitized image data output from the A/D converters 102 ac- 
cording to the cut-off frequency information in the filter 
parameter group output from the parameter selection sec- 
10 tion 320, 

In the filter parameter group shown in FIG. 9, only one 
cut-off frequency is provided for one camera image. This is 
not necessarily suitable when the contraction rate varies 
with the positions on a camera image in the image synthesis. 

15 On the contrary, in the filter parameter group shown in 
FIG. IIB, the cut-off frequency can be changed with the posi- 
tions on a camera image, and thus the non-contracted portion 
of the input image is prevented from blurring. 

The filter parameter group shown in FIG. IIB includes 

20 filtering setting data having one-to-one correspondence with 
the coordinates of pixels in a one-frame camera image. In 
general, filter parameters of adjacent pixels tend to have 
similar values. Therefore, common setting data may be held 
for a plurality of adjacent pixels. For example, common set- 

25 ting data may be held every rectangular area of 8X8 pixels. 
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By this setting, in addition to preventing the non-contracted 
portion of the image from blurring, the data amount of the 
filter parameter group can be reduced compared with the case 
of holding filtering setting data every pixel. 
5 A more significant effect can be provided by combining 

the constructions of EMBODIMENT 1 and EMBODIMENT 2. That is, 
the interlace noise can be eliminated by the technical fea- 
ture of EMBODIMENT 1 and the aliasing distortion can be 
eliminated by the technical feature of EMBODIMENT 1. This 
10 enables generation of a synthesized image with good quality 
even when the vehicle is moving at high speed. 
EMBODIMENT 3 

FIG. 12 illustrates a construction of a vehicle sur- 
roundings monitoring system of EMBODIMENT 3 of the present 

15 invention. This construction is different from the construc- 
tion shown in FIG. 1 in that a brightness correction sec- 
tion 410 is provided between the frame memories 103 and the 
image synthesis section 104 to correct the brightness and 
tint of camera images. A brightness correction parameter 

20 calculation section 420 is also provided to calculate bright- 
ness correction parameters used by the brightness correction 
section 410. 

FIGS. 13A to 13D illustrate an example of an image syn- 
thesis parameter group for generating a synthesized image 
25 from a plurality of camera images. FIGS. 13A and 13B are ex- 
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amples of images taken with two cameras placed on the rear of 
the vehicle. In the illustrated example, an image of the 
area just behind the vehicle is taken doubly by cameras 1 and 
2. FIG. 13C shows an example of a synthesized image, and 
5 FIG. 13D shows an example of an image synthesis parameter 
group. 

In the example shown in FIG. 13D, the parameter element 
at the coordinates (xo, yo) in the image synthesis parameter 
group indicates that a synthesized image should be generated 

10 by combining the pixel at the coordinates (Xii, Yii) in the 
image from the camera 1 and the pixel at the coordinates 
{Xi2/ Yi2) in the image from the camera 2 at a weighting ra- 
tio of 0.7:0.3 • The image synthesis section 104 assigns 
weights to the respective camera images and then adds, to se- 

15 quentially output the results as a synthesized image. This 
enables generation of a smooth synthesized image free from 
formation of a conspicuous juncture due to difference in 
brightness and tint between the camera images, compared with 
the case of generating each pixel of a synthesized image from 

20 a single pixel of one camera image. 

Moreover, in this embodiment, brightness correction pa- 
rameters are calculated using information on an overlap area 
on a synthesized image. 

Specifically, the image synthesis parameter group stored 

25 in the parameter storage section 109 includes overlap area 
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data indicating the coordinates of a pixel in a camera image 
that corresponds to an overlap area on a synthesized image. 
The brightness correction parameter calculation section 420 
receives the overlap area data included in the image synthe- 
5 sis parameter group and calculates a brightness correction 
parameter using brightness and tint data for the pixel in the 
camera image corresponding to the overlap area indicated by 
the overlap area data. 

FIGS. 14A and 14B illustrate an example of a synthesized 

10 image including an overlap area and an example of an image 
synthesis parameter group for this synthesized image. In the 
synthesized image of FIG. 14A, the hatched portions represent 
the coverages of cameras 1 and 2, which overlap at the rear 
of the vehicle forming an overlap area. The image synthesis 

15 parameter group of FIG. 14B includes overlap area data indi- 
cating the coordinates of pixels in the camera images corre- 
sponding to the overlap area, as partly shown in FIG. 14B. 
That isr as the overlap area data it is shown that the pixel 
at the coordinates (Xil, Yil) in the image from camera 1 and 

20 the pixel at the coordinates (Yi2, Yi2 ) in the imaige from 
camera 2 correspond to the coordinates (xo, yo) in the over- 
lap area. 

It is possible to determine whether a given position is 
inside a non-overlap area or an overlap area depending oh 
25 whether the weighting ratio is "1:0" or otherwise. Alterna- 
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tively, a given position may be determined to be inside an 
overlap area if two or more effective camera numbers are 
specified, and inside a non-overlap area if only one effec- 
tive camera is specified. 

The parameter selection section 108 selects the image 
synthesis parameter group according to the vehicle speed de- 
tected by the vehicle motion detection section 107 and the 
vehicle status detected by the vehicle status detection sec- 
tion 200 based on predetermined criteria / sequentially reads 
the selected parameters by interlaced scanning, and outputs 
the results to the image synthesis section 104. The overlap 
area data in the image synthesis parameter group output from 
the parameter selection section 108 is also output to the 
brightness correction parameter calculation section 420. 

The brightness correction parameter calculation sec- 
tion 420 determines the distributions of brightness and tint 
of the respective camera images in the overlap area, from the 
overlap area data output from the parameter selection sec- 
tion 108 and the pixel values read from the frame memo- 
ries 103 by the image synthesis section 104, to calculate 
brightness correction parameters with which the distributions 
of brightness and tint of the respective camera images can 
substantially match with each other in the overlap area. 

The brightness correction section 410 corrects the 
brightness for the pixel values read from the frame memo- 
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ries 103 by the image synthesis section 104 according to the 
brightness correction parameters calculated by the brightness 
correction parameter calculation section 420, and output the 
results to the image synthesis section 104. The image syn- 
5 thesis section 104 generates a synthesized image from the 
camera images of which brightness has been corrected by the 
brightness correction section 410, according to the image 
synthesis parameter group output from the parameter selection 
section 108. 

10 A procedure of calculating the brightness correction pa- 

rameters will be described in a specific example. 

Assume that the brightness correction parameter calcula- 
tion section 420 outputs as the brightness correction parame- 
ters a gain coefficient and an offset coefficient with which 

15 the averages and distributions of the brightness of respec- 
tive camera images can match with each other in the overlap 
area. Also assume that the brightness correction section 410 
corrects the pixel values of the respective camera images 
with a linear equation using the gain coefficient and the 

20 offset coefficient as the brightness correction parameters. 

Assume that a plurality of overlap areas exist. For ex- 
ample, assume that there exist four overlap areas, a first 
overlap area formed by cameras 1 and 2, a second overlap area 
formed by cameras 1 and 3, a third overlap area foinned by 

25 cameras 2 and 4, and a fourth overlap area formed by cam- 
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eras 3 and 4. 

First focusing on the first overlap area, the brightness 
correction parameter calculation section 420 accumulates im- 
ages taken with the cameras 1 and 2 read from the frame memo- 
5 ries 103 by the image synthesis section 104 by one frame each, 
and obtains the averages and distributions of the brightness 
of the images taken with the cameras 1 and 2 in the first 
overlap area. Likewise, for the second to fourth overlap ar- 
eas, the averages and distributions of the brightness of cam- 

10 era images in each of the overlap areas are obtained. 

Next, as brightness correction parameters, a gain and an 
offset are determined so that the averages and distributions 
of the brightness of the images from the cameras 1 and 2 in 
the first overlap area match with each other. For example, 

15 consider the case that the average and distribution of the 
brightness of the image from the camera 1 are 100 and 10, re- 
spectively, and the average and distribution of the bright- 
ness of the image from the camera 2 are 121 and 11, 
respectively. Then, the averages and distributions of the 

20 images from the cameras 1 and 2 will match with each other 
when the gain and the offset are set to 1 and 0 respectively 
for the image from the camera 1, and 10/11 and -10 respec- 
tively for the image from the camera 2 . 

Using the brightness correction parameters determined 

25 for the first overlap area, the average and distribution of 
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the brightness of the Image from the camera 1 in the second 
overlap area are corrected. Note that in this example where 
the gain is 1 and the offset is 0 for the image from the cam- 
era 1, no correction is required for the average and distri- 
bution of the brightness of the image from the camera 1. 
Brightness correction parameters for an image from the cam- 
era 3 are then determined so that the averages and distribu- 
tions of the brightness of the images from the cameras 1 and 

3 match with each other. Likewise, the average and distribu- 
tion of the brightness of the image from the camera 2 in the 
third overlap area are corrected using the brightness correc- 
tion parameters determined in the first overlap area. 
Brightness correction parameters for an image from the cam- 
era 4 are then determined so that the averages and distribu- 
tions of the brightness of the images from the cameras 2 and 

4 match with each other. 

In the calculation procedure described above, proper 
brightness correction may not be obtained in the fourth over- 
lap area. In view of this, as another calculation procedure, 
brightness correction parameters may be calculated temporar- 
ily in the respective overlap areas. For a camera image ex- 
isting in a plurality of overlap areas, the average of the 
temporarily calculated brightness correction parameters may 
be determined as the final brightness correction parameters 
for the camera image. This procedure fails to completely 
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match the brightness and the tint between camera images in 
each overlap area. However, as the entire synthesized image, 
it is expected that all of the overlap areas have no great 
displacement in brightness and tint. 

When a plurality of overlap areas exist, the priorities 
given to the overlap areas to be followed during the calcula- 
tion of brightness correction parameters may be determined 
according to the motion of the vehicle detected by the vehi- 
cle motion detection section 107. 

For example, when the vehicle is moving backward, a high 
priority may be given to an overlap area located at the rear 
of the vehicle on the synthesized image. When the vehicle is 
moving forward, a high priority may be given to an overlap 
area located at the front of the vehicle on the synthesized 
image. In general, the driver tends to pay attention to the 
rear of the vehicle when driving backward and to the front of 
the vehicle when driving forward. Therefore, by giving pri- 
orities, the brightness and the tint can be made uniform in 
the overlap area at the position to which the driver tends to 
pay attention, although they may not be uniform in other 
overlap areas. The resultant synthesized image is therefore 
more natural for the driver. 

As described above, in this embodiment, brightness cor- 
rection parameters are calculated using data on the bright- 
ness and tint of pixels in camera images corresponding to an 
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overlap area, and the brightness is corrected using the pa- 
rameters. This enables generation of a smooth synthesized 
image free from formation of a conspicuous juncture due to 
difference in brightness and tint between the camera images. 

The construction of FIG. 12 includes the brightness cor- 
rection section 410. However, if the caiQeras 101 incorporate 
therein a function corresponding to that of the brightness 
correction section 410, the brightness correction section 410 
may be omitted and thus the system construction can be sim- 
plified. 

The technique for brightness correction is not limited 
to that described in this embodiment, but other techniques 
may also be used as long as the brightness and tint of camera 
images in an overlap area can be made close to each other, 
and substantially the same effect as that described in this 
embodiment can be attained. 

In this embodiment, weighting information in the image 
synthesis parameter group was used to identify an overlap 
area. Alternatively, in place of the weighting information, 
some exclusive information indicating whether or not a given 
position is inside an overlap area may be included in the im- 
age synthesis parameter group. 
EMBODIMENT 4 

In EMBODIMENT 4 of the present invention, in an overlap 
area of the coverages of a plurality of cameras, the selec- 



39 



tion of the camera images used for image syn'thesis and the 
setting of weights for the respective camera images are ap- 
propriately changed. 

The construction of the vehicle surroundings monitoring 
5 system of this embodiment is substantially the same as that 
shown in FIG. 1^ except that the parameter storage sec- 
tion 109, which is the same in construction, includes a plu- 
rality of image synthesis parameter groups different in the 
selection of camera images used for image synthesis in an 

10 overlap area. 

FIG. 15 is a view showing the positions of cameras and 
obstacle sensors as the obstacle detection means 230 on a ve- 
hicle in this embodiment. Referring to FIG. 15, cameras 1 
and 2 are placed on pillars at the rear of the vehicle, where 

15 the camera 1 takes images on the left side of the rear area 
of the vehicle, while the camera 2 takes images on the right 
side thereof. The coverages of the cameras 1 and 2 overlap 
in the rear of the vehicle. Two obstacle sensors are placed 
on the rear of the vehicle to enable detection of an obstacle 

20 in the rear area of the vehicle. 

FIGS. 16A to 16C are views illustrating examples of a 
synthesized image and image synthesis parameter groups for 
generation of the synthesized image. FIG. 16A illustrates a 
synthesized image, and FIGS. 16B and 16C illustrate two image 

25 synthesis parameter groups A and B for generation of the syn- 
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thesized image of FIG. 16A. The image synthesis parameter 
group A (FIG. 16B) and the image synthesis parameter group B 
(FIG. 16C) are used to generate the same synthesized image in 
composition (FIG. 16A) , but use a different camera image for 
5 image synthesis in the overlap area in the rear of the vehi- 
cle. The image synthesis parameter group A uses an image 
from the camera 1 while the image synthesis parameter group B 
uses an image from the camera 2. 

The operation of the vehicle surroundings monitoring 

10 system of this embodiment will be described. 

When the obstacle detection means 230 detects an obsta- 
cle on the right or left side of the rear area of the vehicle, 
it outputs the position of the detected obstacle to the pa- 
rameter selection section ,108. The parameter selection sec- 

15 tion 108 selects an image synthesis parameter group according 
to the position of the detected obstacle and the switch op- 
eration by the user. 

For example, the image synthesis parameter group A is 
selected when the obstacle exists on the left side of the 

20 rear area, and the image synthesis parameter group B is se- 
lected when the obstacle exists on the right side of the rear 
area. The image synthesis section 104 reads camera images 
from the frame memories 103 according to the selected image 
synthesis parameter group to generate a synthesized image. 

25 By the operation described above, in the case shown in 
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FIG. 17 A/ for example/ where an obstacle such as another ve^ 
hide exists on the left side of the rear area, the image 
synthesis parameter group A is selected. This makes the 
juncture formed between the camera images less conspicuous on 
5 the resultant synthesized image as shown in FIG. 17B. If the 
image synthesis parameter group B is selected, the juncture 
between the camera images will run across the obstacle vehi- 
cle on the synthesized image, making the vehicle unnatural 
with the juncture as shown in FIG. 17C. In other words, by 
10 selecting the image synthesis parameter group according to 
the position of an obstacle, the existence of the juncture 
can be made less conspicuous on the synthesized image, and 
thus a synthesized image with natural appearance can be gen- 
erated. 

15 The parameter selection section 108 may otherwise switch 

the image synthesis parameter group according to the motion 
of the vehicle detected by the vehicle motion detection sec- 
tion 107, in place of the output from the vehicle status de- 
tection section 200. By switching in this manner, the camera 

20 image used for image synthesis in the overlap area can be 
switched according to the direction and speed of the driving 
of the vehicle. Therefore, by selecting a camera image from 
a camera of which coverage is closer to the forward area of 
the vehicle, for example, it is possible to reduce formation 

25 of a juncture between camera images in and around an . area 
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mostly viewed by the driver, and thus display a synthesized 
image with which the driver can feel easy in driving. Natu- 
rally, it is also possible to use both the outputs from the 
vehicle status detection section 200 and the vehicle motion 
5 detection section 107 . 

PIG. 18 illustrates another example of the image synthe- 
sis parameter group in this embodiment. The image synthesis 
parameter group of FIG. 18 includes a plurality of sets of 
weighting information for camera images in an overlap area on 

10 a synthesized image. For example, the position at the coor- 
dinates (xO, yO) in the image synthesis parameter group is a 
position at which a pixel at the coordinates (Xil, Yil) in a 
camera image from the camera 1 and a pixel at the coordinates 
(Xi2, Yi2) in a camera image from the camera 2 overlap with 

15 each other. This position includes two sets of weighting 
information, A (0.7:0.3) and B (0.2:0.8) as the weighting 
ratio used during synthesis. 

The parameter selection section 108 selects one of a 
plurality of sets of weighting information included in the 

20 image synthesis parameter group according to the output of 
the vehicle status detection section 200. For example, the 
weighting information A is selected when an obstacle exists 
on the left side of the rear area of the vehicle, and the 
weighting information B is selected when an obstacle exists 

25 on the right side. 
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Thus, by selecting weighting information for image synr 
thesis according to the position of an obstacle , it is possi- 
ble to reduce the unnaturalness of the juncture formed 
between the camera images on a synthesized image and thus 
5 generate a synthesized dLmage with a smoother juncture. 

FIG. 19 illustrates yet another example of the image 
synthesis parameter group in this embodiment. The image syn- 
thesis parameter group of FIG. 19 is equivalent to that of 
FIG. 18 except that one weight reference number indicating a 
10 combination of sets of weighting information is given, in 
place of sets of weighting information, for each pixel. 

In the example shown in FIG. 19, the weight reference 
number for the coordinates (xO, yO) in the image synthesis 
parameter group is "3". By referring to another table, it is 
15 found that the weight reference number "3" indicates that the 
weighting ratio of the pixels to be combined is 0.7:0.3 
(weighting information A) or 0.2:0.8 (weighting informa- 
tion B) . 

While the image synthesis parameter group of FIG. 18 
20 needs to hold four values as weighting information for each 
coordinate position, the image synthesis parameter group of 
FIG. 19 can hold only one weight reference information value. 
This indicates that the data amount of the image synthesis 
parameter group of FIG. 19 is smaller as the number of pixels 
25 in the image synthesis parameter group is greater or the num- 
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ber of combinations of sets of weighting information is 
smaller. 

Thusr the data amount of the image synthesis parameter 
group can be reduced by holding the weight reference number 
5 indicating a combination of a plurality of sets of weighting 
information, in place of holding a plurality of sets of 
weighting information/ for each coordinate position in the 
image synthesis parameter group. 

FIGS. 20A and 20B illustrate other examples of the image 

10 synthesis parameter groups in this embodiment, which are gen- 
erated using a dithering method based on the image synthesis 
parameter group of FIG. 18. 

The image synthesis parameter group A of FIG. 20A is 
generated by a so-called dithering method, where, based on 

15 the camera numbers and the weighting information A for each 
pixel in the image synthesis parameter group of FIG. 18, one 
of the two camera images is selected for each pixel in the 
vicinity of a given pixel so that the ratio of the number of 
pixels using one camera image to that using the other camera 

20 image is close to the specified weighting ratio. Likewise, 
the image synthesis parameter group B of FIG. 20B is gener- 
ated based on the camera numbers and the weighting informa- 
tion B for each pixel in the image synthesis parameter group 
of FIG. 18. 

25 In the weighting information A in the image synthesis 
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parameter group of FIG. 18 ^ at the coordinates (xO, yO)^ the 
weights are 0.7 for the camera 1 and 0.3 for the camera 2. 
In the image synthesis parameter group A of FIG. 20A, the 
camera number is selected so that the ratio of the number of 
pixels using the camera 1 to that using the camera 2 is close 
to 0.7:0.3 in the vicinity of the pixel at the coordinates 
(xO, yO). Thus^ in the example shown in FIG. 20A, although 
the camera 1 is selected for the coordinates (xO, yO) , the 
ratio of the number of pixels using the camera 1 to that us- 
ing the camera 2 is about 0.7:0.3 at and around (xO^ yO). 

The dithering method described above is widely known and 
disclosed in "Television image information engineering hand- 
book"^ ed. by The Institute of Image Information and Televi- 
sion Engineers^ Ohmsha^ Ltd., and the like. Detailed 
description on the method for selecting camera dLmages is 
therefore omitted here. 

The parameter selection section 108 selects the image 
synthesis parameter group A or B according to the output of 
the vehicle status detection section 200. 

Thus, by selecting one from a plurality of dithered im- 
age synthesis parameter groups according to the position of 
an obstacle for image synthesis, it is possible to reduce the 
unnaturalness of the juncture formed between the camera im- 
ages on a synthesized image and thus generate a synthesized 
image with a smoother juncture. 
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The total data amount of the image synthesis parameter 
group A of FIG. 20A and the image synthesis parameter group B 
of FIG. 20B is smaller than the data amount of the image syn- 
thesis parameter group of FIG. 18 by the amount of weighting 
5 information. 

Moreover^ when image synthesis is performed using the 
image synthesis parameter group A of FIG. 20A or the image 
synthesis parameter group B of FIG. 20B, the number of pixels 
read from the frame memories 103 by the image synthesis sec- 
10 tion 104 is smaller, and the amount of calculation by the im- 
age synthesis section 104 is smaller by the addition of the 
weighting, compared with the image synthesis using the 
weight-added image synthesis parameter group of FIG. 18. 

Thus, the use of the image synthesis parameter group 
15 generated by the dithering method provides the effect of re- 
ducing the processing amount required for image synthesis, in 
addition to the effect of reducing the data amount of the im- 
age synthesis parameter group, compared with the use of the 
weighting information-added image synthesis parameter. 
20 In the embodiments described above, the display 106 dis- 

plays interlaced scanning images, and the parameter selection 
section 108, 320 reads an image synthesis parameter group by 
interlaced scanning. The same effects as those described in 
the respective examples can also be obtained by adopting non- 
25 interlaced scanning in both operations . 
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In the embodiment:s described above, each image synt:hesis 
parameter group is data in the form of a table indicating the 
correspondence between the coordinates in camera images and 
the coordinates in a synthesized image. Alternatively, it 
5 may be described in other forms such as a mathematical ex- 
pression including variables and a program as long as the 
correspondence between camera images and a synthesized image 
is clarified. When such forms are adopted, the image synthe- 
sis section may be constructed to obtain corresponding coor- 

10 dinates using an image synthesis parameter group described in 
the form of a mathematical expression including variables or 
a program. Japanese Patent Application No. 10-217261, for 
excunple, discloses a method for generating image synthesis 
parameter groups described in the forms of a mathematical ex- 

15 pression including variables and a program. 

In the embodiments described above, it was assumed that 
the image synthesis parameter group included elements having 
one-to-one correspondence with the coordinates of pixels in a 
one-frame synthesized image and had the same size as that of 

20 the synthesized image. The size of the image synthesis pa- 
rameter group is not necessarily the same as that of the syn- 
thesized image. For example, an image synthesis parameter 
group having a size larger than the synthesized image may be 
prepared, and a portion having the same size as the synthe- 

25 sized image may be cut out from this image synthesis parame- 
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ter group to be used for image synthesis. 

In the above embodiments , the monitoring system and the 
image processor of the present invention were described as 
being applied to cars. Alternatively, they may be applied to 
5 Other types of vehicles such as shipping and airplanes . Oth- 
erwise, cameras may be placed on a non-mobile object to be 
monitored such as a store, a house, a showroom, and the like. 

The positions of a plurality of cameras and the number 
of cameras are not limited to those described above. 

10 The function of the image processor of the present in- 

vention may be implemented by hardware entirely or partially 
using exclusive equipment, or may be implemented by software. 
It is also possible to use a recording medium or a transmis- 
sion medium storing a program programmed to enable a computer 

15 to execute the entire or partial function of the image proc- 
essor of the present invention. For example, a computer as 
shown in FIG. 21 may be used, where the processing means such 
as the image synthesis section and the parameter selection 
section may be implemented by a software program executed by 

20 a CPU 510 and the results may be stored in a ROM 520 or a 
RAM 530. 

Thus, according to the present invention, it is possible 
to reduce the unnaturalness of a synthesized image observed 
when the vehicle is moving and also prevent the reduction in 
25 resolution observed when the vehicle is standing still. In 
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addition r aliasing distortion on a synthesized image can be 
effectively suppressed • Moreover, it is possible to reduce 
the unnaturalness of the juncture formed between camera im- 
ages on a synthesized image. 
5 While the present invention has been described in a pre- 

ferred embodiment, it will be apparent to those skilled in 
the art that the disclosed invention may be modified in nu- 
merous ways and may assume many embodiments other than that 
specifically set out and described above. Accordingly, it is 
10 intended by the appended claims to cover all modifications of 
the invention that fall within the true spirit and scope of 
the invention. 
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